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Stmctural and Linguistic Variables That Contribute to 
Difficulty in the Judgment of Deductive Argunents 
of the Conditional Type 

Jerry R* Shipnan 
Alabama A» & Uoiversity 

ABSTRACT ^ 

Eighteen structural and linguistic \Jriables were id^tified, defined^, 
and used in an attempt to determine which variables would account for a 
significant amount of the observed variance in the difficulty level of items ^ 
on two different conditional reasoning tests administered to high school and 
college students. Using regression analysis involving these structural and 
linguistic variables, a tentative subset of variables was identified which 
accounts for a significant amount of the variance in the proportion of correct 
judgments of verbal simple deductive arguments of the conditional type* 



STRUCTURAL AND LINGUISTIC VARIABLES 
THAT CONTRIBUTE TO DIFFICULTY IN THE 
JUDGMENT OF DEDUCTIVE ARGUMENTS OF 
THE CONDITIONAL TYPE 

A number of receat studies by Jerman and others (e^g*^ Jerman and Rees, 
1972; Jerman^ 1973 b; Krushinskl^ 1973; Beardslee and Jerman, 1973^ 1974; 
Cook, 1973^ 1974) have focused on the influence of structural and linguistic 
variables on the relative difficulty of verbal and/or computational arithmetic 
and algebra problems. Jansson (1974) conducted an exploratory Investigation 
In an attempt to identify those structural and linguistic variables which 
strongly influence the relative difficulty of subjects handling the judgment 
of simple deductive arguments in verbal form. The purposes of the present 
study were to refine and further investigate the variables explored in Jansson* s 
study 9 identify and define in a clear and explicit way a set of new -relevant - 
structtural and linguistic variables, determine which set of these variables 
would account for a significant amount of the obseirved variance in the propor-* 
tion of correct judgments made by high school and college subjects on tests 
involving verbal simple deductive arguments of a conditional type of reasoning ^ 
and attempt to shed light on a possible way curriculum developers and teachers 
can control the relative difficulty of such deductive arguments ^en preparing 
instructional materials* 

A simple deductive argument in the present investigation refers to a chain 
of reasoning Involving three statements or propositions, where the first two 
propositions are premises (assumed to be true) and the third one is the conclu* 
sion drawn from the premises* If at least one of the premises is a conditional 
statement^ then the argument is of the conditional type; Ennis and Paulus (1965), 
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in the Cornell Critical Thinking Project, viewed the judgment of Quch arguments 
as a significant component of the broader concept of critical thinking, which 
in turn was defined in terms of the notion of the "correct assessing of 
statements." ' ^ 

A nuniber of other researchers (eog». Hill, 1961; O'Brien and Shapiro, 1968; 
Roberge and Paulus, 1971; O'Brien, Shapiro, and Reali, 1971; Roberge, 1969, 1970; 
Gardiner, 1965; Shlpman, 1974; Tripp, 1974; Jansson, 1973; DlNapoli, 1974; Rlzza, : 
1974), both within and outside the conceptual framework of Eonis and Paulus and 
the Cornell Critical Thinking Project, have studied the logical skills of subjects 
of a wide range of ages over such variables as type of logic reasoning (i.e., ^^I" 
class and conditional), principle of reasoning (such as, modus ponens, contra-^^ ^ 
positive, converse, inverse, and transitivity), age, sex, content and context of / 
arguments, negations in agruments, subject aptitude and their interactions. ^In 
an overall sense, the variables investigated in these studies dealt with structure 
of the problem itself) and learner aptitude->interaction characteristics with 
regard to correctly handling the judgment of simple deductive arguments in verbal 
form. 

While the studies in this framework have shed some light on the nature of 
these structural and linguistic variables and learner characteristics, no clear 
evidence is yet available so that we can say to curriculum developers and teachers 
which of these variables and/or learner aptitude-»interaction factors minimize the . 
difficulty level in verbal simple deductive arguments, 

Jansson (1974) made an initial attempt to employ a new^ approacfiy^rltnear 
regression analysis tcchniques~to the study of structural and linguistic variables 
as predictors of the relative difficulty in verbal simple deductive arguments. ' 
Of course, this is not to imply that learner aptitude-interaction factors do not 
come into play, but until clear evidence is available concerning such factors, we 
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believe that this new approach will prove to be a more significant avenue for 
research and currlcular /instructional development • This regression approach 
has been used by Jerman and his associates in studies of structural and 
llngusltlc variables as predictors aft- difficulty in the areas of verbal and 
coQputatlonal problem solving in arithmetic and algebra* The approach In the 
present study was to employ linear regression techniques to identify those 
structural and linguistic variables that account for a significant amount of the 
observed variance in the proportion of correct judgments of verbal simple condi-> 
tlonal arguments • 

II. Method 

The Linear Regression Model 

The linear regression model employed in the present study is the same one 
specified and outlined in Jansson's (1974) study* In an attempt to be complete, 
the model and its notation and rationale are repeated here. Let the Jth 
variable of problem 1 be denote by v^j« The corresponding weight assigned to the 
jth variable is denoted by aj* Let p^^ be the observed proportion of correct 
responses on problem 1 for a given population sample. The purpose of the model 
is to predict P^^ i'or each problem 1, Thus, in terms of the variables v^j and 
their associated weights a^, the linear regression model is given by 

Pi - ^ ^j^ij + % <1> 

Since the estimated weighting and values for the vari^^bles are combined to 
predict p^, this model may not preseirve probability. So in order to insure that 
the p^*s will always lie between 0 and 1, the usual practi«:o has been to make 
. the following logarithmic transformation and define a new variable z^. 

- log [a-Pi)/p^] (2) 
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Thus 9 the regression model becomes 

H « 2 a^v^^ + a^. (3) 
In the case where the observed is either 0 or 1, we define as follows: 

log (2n. -1) for p « 0 

^i - ^ 

log [l/(2n^ -1)] for p^ « 1, 

where n^ denotes the total number of subjects responding to problem i# According 
to Jerman and Rees (1972; p*307): 

"The reason for putting 1-pj in the numerator of equation (2) is 
to make the variables z± increase monotonically in dif ficulty© 
It is desirable that the model reflect an increase in difficulty 
level rather than inversely as the magnitude of the variables 
Vjj increases*" 

This model, together with the transformation as indicated above, were adhered to 
in the present study. 

^oi'inition o f Varigtb les 

In the present study 18 structural and linguistic variables were usedo 
Eleven of these variables, some of which were slightly refined by the writer, 
were originally developed in Jansson's (1974) study o The remaining seven 
variables were newly developed based on suggestions for further investigation 
made in Jansson's study, as well as implications made in studies by Roberge (1971), 
Shipman (1974), and Tripp (1974) o 

The variables used in the present investigation are defined as follows, with 
the newly developed variables beginning with variable 12. 

Xj^: Principles of Inference (PRINFR) 

Coded: 1 for modus ponens; 2 for transitive; 3 for contrapositive; 
4 for inverse; 5 for converse 

Validity of the Argument (VALIDY) 

Coded: 1 if valid; 2 if invalid; 3 i* "can't tell" 



X3: Content type (CONTNT) 

Coded: 1 for concrete-familiar; 2 for suggestive; 3 for abstract 
X^: Sentence length (SENTL) 

Coded: One count for each word in the total argument* 
X^: Total number of words in premise 1 (HRDPl) 

Coded: One count for each word in premise 1 

Xg: Average word length (AHRDL) 

Coded: Average word length in number of letters of the total argument o 

That is, the total ntmiber of letters in the argxxment, not counting 
punctuation y divided by the total ntmiber of words in the argtmient. 

Xji Ratio of nxmiber of words in premises (WP1P2) 

Coded: Value of the ratio of the total number of words in premise 1 to 
the total nxmiber of words in premise 2. 

Xg: Total number of negations in the argimient (TOTNGS),.' 

Coded: One count for each negation in the argument. 
Xqi Number of negations in premise 1 (NE6P1). 

Coded: One count for each negation in premise 1. 
Xj^O* Number of negations in premise 2 (NEGP2) 

Coded: One count for each negation in premise 2. 
Xj^^i Number of negations in conclusion statement of the argument (NE6C). 

Coded: One count for each negation in conclusion statement of the argument 
Xj^2* Total number of words in the conclusion statement of argument (WRDC) 

Coded: One count for each word in the conclusion statement » 
Xj^ji Average word length in premise 1 (AWRDLPl). 

Coded: Average word length in number of letters of premise 1» 
Xj^^: Average word length in premise 2 (AWRDLP2) 

Ceded: Average word length in number of letters of premise 2. 
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X^^z Average word length of antecedent In premise 1 (AWDIA)«. 

Coded: Average word length In number of letters in antecedent of premise 1 
That is, total number of letters in the antecedent divided by the 
total number of words in the antecedent* 

X,^: Average word length of consequent in premise 1 (AWRDLC)* 
16 

Coded: Average word length in number of letters la the consequent of 
premise 1» . * 

yi^jt Negation in the antecedent of premise 1 (NEGPIA) 

Coded: 1 if negation exists; 0 If negation does not exist • 

Xj^g: Negation in the consequent of premise 1 (NEGPIC) 

Coded: 1 if negation exists; 0 if negation does not exist* 

T he Conditional Reasoning Te sts 

Two testing instruments, the Shipman-Trlpp Conditional Reasoning Test (Pprm 
C123-4), a 23-item test which was developed and used In studies by Shj||M((l974) 
and Tripp (1974), and the Shipman Conditional Reasoning Test (Form CFS-5), a 30- 
item test which was developed and validated for the present study, were used to . 
gather data* Each of these tests stems from the work of Paulus (1967) and 
Roberge (1970) and the conceptual frarawork of the Cornell Critical Thinking 
Project, 

The items, or arguments, on each test are designed to evaluated subjects' 
abilities to judge the validity of selected principles o| deductive reasoning of 
the conditional type in verbal form and In different content dimensions* Itm 
format on each test is common as follows; 

Suppose you know that 

Premise lo 

Premise 2* 
Then would this be true? 

CcnclusioUo 
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For each item, three responses were possible for indicating its validity status 
and were defined as follows; 

1* YES It must be true. 

2. NO . It can't be trueo 

3. MAYBE It may be true or it may not be trueo You were 

not told enough to be certain whether it is "YES" 
or "NOo" 

The Principles o f Reasoning 

In the present investigation, five basic principles of conditional reasoning 
were of interest, fiach of these principles, along with its symbolic form, a 
sample verbal form and its validity status, is exhibited in Table 1. The Shipman 
test covers all of these principles, while the Shipman-Tripp test covers all 
except the modus ponens principleo 

Insert T^ble 1 about here 



The Content Dimension 

In the literature (eogo Wilkins, 1928; Tripp, 1974), the content dimension 

of logical arguments is divided in three segments: (1) concrete-familiar, 

(2) suggestive, and (3) abstract o Concrete-familiar items are those in which the 

conclusion of the argument possesses a neutral truth value with the vocabulary 

Aroughout the argument being familiaro If at least one statement in the argument 

contradicts common knowledge, then the argument is labeled suggestiveo Samples of 

concrete-familiar items are given in Table lo An example of a suggestive item 

is as follows: 

If frogs can not hop, then cats can singo 
Cats can sing. 

Therefore, frogs can not hopo " ' 
Both tests contain concrete-familiar and suggestive items, which were used in the 
present investigation. 

The third segment of the content dimension involves abstract' or nonverbal 
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symbolism which was not considered a part of the present study. 



HI* Procedures 



Each item on both tests was identified and quantified with respect to the 
18 structural and linguistic variables. These sets of information formed the 
data base for the independent or predictor variables in the sti*dy« 

The tests were administered at a predominately black university to lOthy 
11th 9 and 12th grade Upward Bound students, freshman elementary education majors, 
and freshman general mathematics students with little or no formal training in 
logical thinking* The Shipman-^Tripp test was administered to 115 subjects, of 
which 54 were Upward Bound students. The Shipman test was administered to 70 
freshman general mathematics students. In the administration of each test, the 
examiner directed the students to complete each item* It was emphasized that- 
the tests ware not speed tests* 

The data were subjected to linear regression techniques. Specifically, 
using a step-wise linear regression computer program, seperate linear regression 
analyses with a common logarithmic transformation as described earlier were 
applied to each of the following data sets: 

1. Shipman-Tripp Conditional Reasoning Test - College subjects (S-T (C)) 

2. Shipman-Tripp Conditional Reasoning Test - High school subjects (S-T (H)) 

3. Shipman-Conditional Reasoning Test (SCRT), 

A linear regression analysis was performed on a subset of each of these data sets. 
The computer program in each instance was designed to indicate what specific 
variables entered the regression model and in which order* 




IV. Results ^ 



Summary data for the tests are presented In T::ble 2. In Table 3 the 
entry order of the 18 variables for each test together with multiple R values 
is given* 



Insert Table 2 & 3 about here 

A deatiled look at the ret^ults of the regressiott analyses is considered below 
with respect to each test* 

1. Shlpman-*Tripp Conditional Reasoning Test 

College subjjects* The variables which entered and their entry order for 
this test over college subjects are displayed In the first column of Table 3« 
It is apparer^t that variable 2^ VALIDT, entered firsts and it accounted for a 
major portion of the observed variance linearly 45 percent). A total of six 
steps in the regtession analysis was needed to account for approximately 73% 
of the total variance in the observed probability correct* Variable 1^ FRINFR^ 
unexpectedly entered the regression at step 5« 

In Table 4 the regression coefficien::s along with computed t«^alues at the 
junction where all entering variables have actually entered the regression are 
given* Notice that variable 5» NE6P1» was the only variable significantly 
different from zero and it entered the regression at step 4» 



Insert Table 4 about her-^ 



Hljj[h j^clyol subjects * In colimtn three of Table 3 the VALIDY variable is shown 
to enter the regression analysis first for the Shlpman-Trlpp test over high school 
subjects* However^ in this case^ only 38 percent of the observed variance In the 
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probability correct is accotmted for* Two of the same variables, WRDPl and 
PRINFR, which entered the S-T (C) regression also entered within the first 
six steps of the S-T (H) regression. The amount of total variance accounted 
for in the observed probability correct at step 6 was approximately 70 percent. 

The regression coefficients with t-values calcxilated at the point where 
all entering variables have entered the regression are exhibited in column two 
of Table 4« None of the variables were significantly different from zero at a 
desirable level • 

2. Shipman Conditional Reasoning Test 

For this test the order of entry and corresponding R values of the variables 
are presented in column 6 and 7 of Table 3, while the regression coefficients 

and their t-values are presented in Table 5. 

♦ 

Insert Table 5 about here 

In step 1 of the regression, variable 1, PRINFR, entered and accoimted for 
more than 65 percent of the observed variance in the probability correct o Tlie 
addition of more variables, 3,/4, It, 18, and IS, increased R square to approxi- 
matily 84 percent. Variable A, VALiDY, did not enter the regression until step 
11* At the point where all variables entered only the PRINFR variable was 
statistically significant o 

3. Results of Other Analyses on Tests 

Using a selected subset of the original variables from each of the data sets, 
some further analyses were performed. In the case of the Shipman-Tripp test over 
each group of subjects, the variables which entered tiae orte&tal calculations in 
the first 10 steps were used as .a subset. This was done because at step 10 the 
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R square value, in each Instance, was approximately 81*5 percent. The restilting 
regression coefficients, t-values and entry orders of the variables appear in 
T^ble 6. For both calculations the VALIDY variable came in first and was 
statistically significant, while different variables involving the location. of 
negations entered at step 2 as was true in the first calculations. 

Insert Table 6 and 7 about here 

For the Shipman test (Table 7) the variables ^^ch entered in the first 
five steps of the first calculation were selected for further investigation. 
The PRINFR variables remained as the major contributor to the variance accounted 
for (65 percent) in the regression. The addition of the other four variables 
increased the value of R square to more than 83 percent* 

The correlation matrices for the data sets appear in Tables 8 and 9. Notice 
that for both test r]^^2 fairly high in a positive direction. An obvious 
reason for this is the simlliar nature of the definitions of the variables 
involved. Other similarities may be noted, but will not be discussed here. 

Insert Tables 8 and 9 about here 

A number of studies, including Jansson's (1974) study, have shown that the 
different principles of inference influence subjects' abilities to judge simple 
conditional arguments in verbal form» In the present study this finding is not 
ccmipletely substantiated. However, in an overall sense, the present study seems 
to indicate that the PRINFR variable is relatively strong in its contribution to 
the observed variance in the proportion of correct judgments. 

•V. '\ 

Some of the other findings in the present study vary with those findings 
made in Jansson's (1974) study. This, of course, is to be expected since some 
of the newly defined variables nudged ahead of some of the original defined 
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variables and perhaps a few different learner-aptitude interaction variables 
came into play. In spite of this, an inspection of R square values in the 
present investigation suggests that a few new important variables have been 
uncovered. 

The present study suggests that the variables involving selected locations 
of negations in the argument, average word length in various parts of the 
argument, validity, and content may be important and deserve further study. 
Puther study must also be performed on several different subsets of the variables 
defined here. In addition, study should be conducted to examine performance 
over a range of age/grade levels on the tests used in this study. 

In summary we can conclude that while the present study is important in 
that it approaches or points to a procedure for finding structural and linguistic 
variables which Influence the relative difficulty of subjects handling the 
judgment of verbal simple deductive arguments of the conditional type via regre- 
ssion analysis methods, we are not yec in possession of satisfactory difficulty 
predictors, nor can we prescribe for the curriculum developer and teacher 
appropriate conditional logic exercises (arguments) for secondary school and 
college subjects. Yet, this study does move researchers a bit closer to finding 
stable difficulty predictors which can be used in constructing appropriate 
mathematics materials in the area of conditional logic. 
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TABLE 1 

Basic Principles of Conditional Reasoning 











Principle 


Valid 


Symbolic form 


Sample concrete verbal form 


1 


: YES 


If P, then Q. 
* • 


If the truck is a Ford, then it 

Dcxongs CO jonn* 
The truck is a Ford^ 
♦ the truck belongs to John« 


2 

Contrapositive 


■ YES 


If P, then Q 
• Not Q. 

not P. 


If the car is black, then it is 

a police car* > 

The car is not a police car* 

• the car is not black* 
} • •» 


3 

Converse 


NO 


■ If P, then Q. 
Q- 

• • « P« * 


^ If the shirt is green, then I 

will weat it* 

I will wear the shirt* 

* the shirt is green* 
♦ ♦ f , . . . 


4 

Inverse 


/ NO 


If P, then Q. 
\ Not P. 
^ not Q. 


If the cat is grey, then his 

name is Shaft* 
Tlie cat is not grey* 

* the cat!s name is not Shaft* 


5 

Transitivity 


: YES 


. If P, then Q. 
; If Q, then R* 
; if P; then R. 


If it is Jim, then it is time 
to go* 

If it is time to go, then call irte* 
if it is. Jim, then call me* 
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TABLE 2 
Summary of Test Results 













Number of 


Number of 


Test 


X 


0 


Test Items . 


Subjects 


Shipman-Tripp (C)* 


7.77 


3.11 


23 


61 . 


Shipman-Tripp (H)** 


6.83 


2.22 


23 




Shlpman 


12.80 


4.19 


30 


70 



*College subjects tested • \. iff 

**High school subjects tested* ' V* 
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TABLE 3 

Entry Order and Corresponding Multiple R Values for the Variables 



Step 


S-l 


' (c) 


S- 


r— 

■T (H) [ S 


CRT . 




^Variable 
Entered , 




Variable 
Entered 


R 


V a i. xct X c 

Entered 


R 


1 


VALIDY 


.662 


VALIDY 


.619 


PRINFR 


.810 


2 


NEGPIC 


.691 


NEGP2 


.691 


CONTNT 


.846 • 


3 


WRDC 


.739 


PRINFR 


.741 




.869 


4 


WRDPl 


.773 


NEGC 


.784 


AWRDLC 


.905 


5 


PRINFR 


.810 


AWPDLPl 


.822 


NEGPIC 


.912 




AWRDU 


.853 


. WRDPl 


.834 


AWRDLA 


.916 


7 


NEGPl 


.862 


SENTL 


.863 


WP1/P2 


.919 


'8 


NEGC 


;883 


■ WP1/P2 


.884 


WRDC 


.924 


9 




.887 


' WRDC 


.889 


NEGP2 


.928 


10 


W1/V2 


.904 


. AWRDLC 


.905 


NEGC 


.935 


11 


AWRDLP2 


.926 


' AWRDLA 




VAT TDY 


.939 


12 


AWRDLC 


.930 


' CONTNT 


.915 


AWl^DLPl 


.944 


13 


AWRDLPl 


.953 


AWRDLP2 


.919 


AWRDL 


.945 


14 


NEGP2 


.958 


. AWRDL 


.926 


WRDPl 


.946 


15 


AWRDL 


.960 


NEGPIA 


.943 


SENTL 


.947 


16 


CONTNT 


.963 


TOTNGS 


.944 


NEGPIA 


.947 


17 


TOTNGS 


.963 


NEGPIC 


.944 


TOTNGS 


.949 



21 



19 



TABLE 4 



Regression Coefficients, Standard Errors of Regression Coeffients, 

and computed t-values 



Variable 


S-T (C) • ■ 


S-T ao 


Mo* 








t 


Coeff * 


. S.E. 


t 


1 


PRINFR 


.325 


.162 


2.001f- 


.266 


.149 


1.784C 


2 


VALIDY 


.142 


.204 


.695 


.133 


.168 


.708 


3 


CONTNT 


.121 


.182 

• 


.666 


.185 


.167 


1.106 


4 


SENTL 


.109 


.086 


1.266 


.037 


.07? 


.462 


5 


WRDPl 


-.292 


.109 


-2.669^ 


-.146 


.101 


-1.446 


/a 


WRDC 


-.025 


.136 


■ -.186 


.018 


.125 


.145 


4 


AWKDL 


.108 


.155 


.693 


.132 


.143 


.924 


13 


AWRDLPl 


1.521 


.948 


1.604 


.995 


.873 


1.139 


/♦ 


AWBDLP2 


.031 


.185 


.167 


.189 


.171 


1.107 


19 


AWBDLA 


-.792 


.686 


-1.154 


-.826 


.632 


-1.308 


li 


AWRDLC 


-1.250 


.728 


-1.716 


-.992 


.671 


-1.479 


a 


MP1/P2 


.359 


.504 


.710 


-.037 


.465 


-.079 


9 


TOTNGS 


.107 


.429 


.250 


-.075 


.395 


-.189 




NEGPl 


.269 


.411 


.653 








10 


NEGP2 


.052 


.496 


.104 


-303 


.457 


-..663 


u 


NEGC 


-.297 


.428 


-.695 


.105 


.394 


.265 


17 


NEGPIA 








.304 


.379 


.803 


18 


NEGPIC 


-.643 


.324 


-1.988 


.035 


.485 


.073 


< .01 


bp < 


.05 


■< .1 









ERIC 
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TABLE 5 



Regression Coefficients, Standard Errors of 
Regression Coefficients, and Computed t<*>Value8 

Shipman Test 



Variable 


Coeff. 


.... 

SoEo 


t 




[(». 


Name 






-PRINFR 


.271 


.072 


3,789* 




t t 


VALIDY 


,095 


• 105 


.907 






CONTNT 


.373 


.195 


1.916 






SENTL 


.027 


•054 


.511 




• 


WRDPl 


-.057 


.081 


-,704 






WRDC 


.015 


.086 


.180 




> 


AWRDL 


.369 


.480 


.769 






AWRDLPl 


-1,536 


1.320 


-1.164 




• 


AVRDLP2 


-,258 


•114 


-2,261^ 






AWRDLA 


£ CO 

.653 


.560 


1.168 






AWROLC 




.572 


f / tic ' 

1.485 






WP1/P2 


O Cfl 

.Z59 


,145 


lo786 






TOTNGS 


-.217 


.292 


-,746 






NEGPl 












NEGP2 


,417 


,312 


1.338 






NEGC 


.010 


.309 


.034 






NEGPU 


.220 


.288 " 


.766 






NEGPIC 


.205 


.303 


.676 


*P < ,01 


^P < ,05 
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